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INTERIOR-POINT  ALGORITHMS,  PENALTY 
^METHODS  AND  EQUILIBRIUM  PROBLEMS 


HANDE  Y.  BENSON,  ARUN  SEN,  DAVID  F.  SHANNO,  AND  ROBERT  J. 

VANDERBEI 

Abstract.  In  this  paper  we  consider  the  question  of  solving  equi¬ 
librium  problems — formulated  as  complementarity  problems  and, 
more  generally,  mathematical  programs  with  equilibrium  constraints 
(MPEC’s) — as  nonlinear  programs,  using  an  interior-point  approach. 
These  problems  pose  theoretical  difficulties  for  nonlinear  solvers, 
including  interior-point  methods.  We  examine  the  use  of  penalty 
methods  to  get  around  these  difficulties,  present  an  example  from 
game  theory  where  this  makes  a  difference  in  practice,  and  pro¬ 
vide  substantial  numerical  results.  We  go  on  to  show  that  penalty 
methods  can  resolve  some  problems  that  interior-point  algorithms 
encounter  in  general. 


1.  Introduction 

Devising  reliable  ways  to  find  equilibria  of  dynamical  systems  has 
attracted  much  interest  in  recent  years.  Many  problems  in  mechanics 
involve  computing  equilibria,  as  do  problems  in  economics  dealing  with 
general  equilibrium  or  game  theory.  In  finance  certain  options  pricing 
problems  may  also  be  formulated  as  equilibrium  models.  What  is  of 
interest  here  is  that  equilibrium  problems  can  be  viewed  in  a  natural 
way  as  mathematical  programming  problems.  In  this  context,  they 
are  usually  referred  to  as  complementarity  problems.  Very  briefly,  a 
complementarity  problem  is  to  find  a  vector 

x  G  Mn  s.t.  0  <  x  _L  g(x)  >  0, 
for  a  function  g  :  Rn  — ►  Rn,  and  x  J_  g(x)  means  xTg(x)  =  0. 
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For  a  survey  of  common  applications  of  complementarity  problems, 
see  [7].  The  options  pricing  example  is  described  in  [12].  For  a  survey 
of  common  algorithmic  approaches  to  solving  these  problems,  see  [8]. 
Most  of  the  methods  in  [8]  are  designed  specially  for  solving  comple¬ 
mentarity  problems.  Many  of  them  involve  solving  a  system  of  non¬ 
smooth  equations,  while  some  of  them  use  a  simplex-type  pivoting  tool. 
By  and  large,  much  of  the  work  involving  these  methods  has  been 
theoretical;  reliable  practical  implementations  have  been  rare.  Thus 
many  real-world  equilibrium  problems  have  remained  intractable,  leav¬ 
ing  room  for  new  approaches. 

Recently  one  popular  approach  has  been  to  view  complementarity 
problems  as  a  type  of  nonlinear  program.  This  idea  had  not  been  re¬ 
garded  very  favorably  in  the  past,  since  nonlinear  codes  were  not  very 
robust,  and  equilibrium  problems  pose  theoretical  difficulties  for  non¬ 
linear  programming.  However,  the  advent  of  reliable  nonlinear  codes 
in  recent  years  has  made  it  possible  to  reconsider  this  approach.  More¬ 
over,  the  use  of  general  nonlinear  solvers  makes  it  possible  to  add  an 
objective  function  to  the  problem,  thus  bringing  under  one  framework, 
complementarity  problems  and  the  more  general  mathematical  prob¬ 
lems  with  equilibrium  constraints  (MPEC’s). 

The  rest  of  this  paper  is  organized  as  follows:  In  Section  2,  a  gen¬ 
eral  interior-point  code  for  nonlinear  programming  in  described.  In 
Section  3,  we  get  into  the  details  of  complementarity  problems  and 
MPEC’s,  and  the  theoretical  difficulties  they  might  pose  for  nonlinear 
solvers.  Section  4  describes  a  way  to  resolve  these  difficulties,  namely 
penalty  methods  (also  known  as  constraint  relaxation).  Section  5  gives 
an  example  from  game  theory  which  highlights  why  these  theoretical 
considerations  can  actually  matter.  Section  6  describes  an  implementa¬ 
tion  of  a  penalty  method  using  LOQO,  and  describes  numerical  results 
for  two  standard  testsets  using  this  implementation.  In  Section  7  we 
show  that  penalty  methods  can  resolve  two  problem  that  interior-point 
methods  such  as  LOQO  can  encounter  in  general  (not  just  in  the  context 
of  MPEC’s):  jamming  and  infeasibility  detection. 

2.  An  Interior-Point  Method:  A  Brief  Description 

The  following  is  only  a  brief  description  of  an  interior-point  method 
for  nonlinear  programming,  for  details,  see  LOQO  [20]  or  IPOPT  [21]. 
Another  interior-point  method,  KNITRO  [4]  uses  the  same  paradigm, 
but  its  implementation  is  very  different.  For  ease  of  notation  we  de¬ 
scribe  the  algorithm  for  problems  with  inequality  constraints  only  (the 
situation  is  much  the  same  for  equality  constraints  again,  see  [20]). 
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Thus  the  problem  is 


min  f(x) 
s.t.  hi(x)  >  0, 


i  =  1, . . .  ,m, 


where  we  assume  that  /(x)  and  each  of  the  hi{x)  are  twice  continu¬ 
ously  differentiable  and  x  €  Rn.  Adding  nonnegative  slacks,  w,  to  the 
inequality  constraints  in  (1)  and  putting  these  slacks  in  a  logarithmic 
barrier  term  in  the  objective  function,  the  problem  becomes: 


(2) 


min 

s.t. 


m 

f{x)  -  d^log(Wi) 

i= 1 

h(x)  -  w  =  0. 


Letting  the  vector  A  represent  the  Lagrange  multipliers  for  (1)  the 
first-order  optimality  conditions  for  the  problem  are 

Vf(x)-A(x)T  A  =  0, 

(3)  -8e  +  WAe  =  0, 

h(x)  —  w  =  0. 

Here  e  is  the  vector  of  all  ones,  A{x)  is  the  Jacobian  of  the  vector  h(x), 
and  W  and  A  are  the  diagonal  matrices  with  the  coordinates  of  the 
vectors  w  and  A  on  the  diagonals,  respectively. 

We  use  one  step  of  Newton’s  method  to  generate  a  triple  (x,w,  A) 
which  solve  the  linearization  of  (3)  for  each  value  of  8,  where  the  se¬ 
quence  of  theta's  monotonically  decreases  to  0  [20]  as  we  approach  the 
optimal  solution.  We  set 

m 

H(x,  A)  =  v2/(x)  -  £>v2M*), 

1 

<x  =  V/(z)  -  A(x)T A, 

7  =  8W~le  -  A, 

p  =  w  -  h( x). 

We  then  find  the  directions  given  by  Newton’s  method  by  solving 
the  reduced  KKT  system: 


—H(x,  A) 

A(x)t 

Ax 

a 

A(x) 

WAT 1 

AA 

_  p  +  WAr'i . 

Aw  is  computed  according  to: 

Aw  =  WA-1(7  —  AA). 
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If  the  algorithm  is  currently  at  the  point  (xk,wk,Xk),  the  new  point 
(xk+1,wk+1,  Afc+1)  is  obtained  as  follows:  Let 


X 

Ax(a:, 

w,  A) 

T(x,w,  A)  = 

w 

+  a(x,w,  A) 

Aw(x 

w,  A) 

_  A  _ 

AA(x, 

w,X)  _ 

Then 


Wk+ 1 

Afe+1 


=  T(xk,wk,Xk) 


where  a  is  chosen  to  ensure  that  wk+1  >  0,  Afc+1  >  0,  and  that  either 
the  barrier  function  or  the  infeasibility  is  sufficiently  reduced  with¬ 
out  making  the  other  much  worse  (see  [3]  for  details).  Note  that  this 
ensures  that  multipliers  corresponding  to  inequality  constraints  will 
always  remain  positive,  a  point  that  will  be  relevant  later. 

LOQO  decides  that  a  solution  is  optimal  when  the  primal  infeasibility, 
p,  the  dual  infeasibility,  a,  and  the  duality  gap,  s^,  are  sufficiently 
small. 


3.  Complementarity  Problems  and  MPEC’S 

As  mentioned  in  the  introduction,  the  complementarity  problem  is 
to  find  a  vector 

x  €  Rn  s.t.  0  <  x  -L  g(x)  >  0, 

for  a  function  g  :  Rn  — *■  Rn,  and  x  J_  g(x)  means  xTg(x)  —  0.  Note 
that  the  nonnegativity  also  implies  Xigi(x)  =  0, i  =  1, . . .  ,n.  We  will 
refer  to  this  way  of  writing  it  as  the  pairwise  formulation  (and  the 
first  way  as  the  summation  formulation).  For  now  we  will  use  the 
pairwise  formulation,  but  the  two  formulations  can  have  differing  effects 
on  algorithmic  performance  (more  on  this  later). 

The  more  general  mathematical  program  with  equilibrium  constraints 
(MPEC)  may  be  written  as  the  NLP 

min  f(x) 

-4.  s.t.  hi(x)  =  0,i  =  =  gi(x),i  =  l...n, 

^  ’  x,  y  >  0,  xuyi  <  0,  i  =  1,  ...n, 

x  =  (xi,x2)  €  R n+l,g  :  Rn+l  -» IT. 

The  first  question  which  arises  is,  why  does  this  problem  deserve 
special  attention?  The  reason  is  that  (4)  has  a  special  structure  which 
is  guaranteed  to  cause  theoretical  difficulties.  Specifically,  it  is  a  well- 
known  fact  that  the  set  of  optimal  Lagrange  multiplier  solutions  is 
always  unbounded  (see,  for  instance,  [15]),  when  a  standard  constraint 
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qualification,  known  as  MPEC-LICQ,  holds  (MPEC-LICQ  simply  says 
that  standard  NLP  constraint  qualifications  are  violated  only  because 
of  the  complementarity  constraints-  see  [15]  for  a  formal  definition).  It 
is  easy  to  see  that  this  remains  true  for  the  multipliers  that  are  gener¬ 
ated  when  applying  an  interior-point  method.  The  next  thing  to  realize 
is  that  an  unbounded  set  of  Lagrange  multipliers  at  the  optimum  can 
cause  problems  for  many  traditional  nonlinear  solvers,  and  especially 
for  an  interior-point  code.  This  is  due  to  the  fact  that  interior-point 
methods  try  to  find  the  center  of  the  face  of  optimal  solutions  for  any 
set  of  variables  (primal  or  dual).  Thus  an  unbounded  set  of  optimal 
multipliers  can  mean  that  the  multipliers  get  larger  and  larger  as  a 
solution  is  approached.  This  in  turn  can  cause  serious  problems  for  the 
algorithm,  as  seen  by  simply  noting  that  one  of  the  equations  in  the 
KKT  system  (3)  is 

WK  =  Be, 

which  means  that  as  multipliers  get  large,  slacks  must  get  small.  If 
the  multipliers  become  large  while  6  is  still  significantly  greater  than 
0,  then  slacks  might  have  to  become  small  prematurely.  Since  the 
slacks  are  always  positive,  very  small  slacks  can  result  in  excessively 
shortened  steplengths,  slowing  progress  of  the  algorithm,  or  making  it 
stall  completely. 

The  problem  with  unbounded  multipliers  just  described  exists  even 
if  every  pair  in  a  solution  is  strictly  complementary  (meaning  that 
exactly  one  member  of  each  pair  is  0).  For  interior-point  methods  the 
problem  is  compounded  when  the  solution  is  not  strict.  We  can  see 
this  as  follows: 

We  set  up  the  interior-point  paradigm  for  (4),  by  adding  nonnegative 
slacks  to  the  inequality  constraints,  and  putting  them  in  a  barrier  term 
in  the  objective  function: 

n 

f(x)  -  0  ^  log  Wi 
1=1 

hi(x)  =  0  ,i  =  1  =gi(x),i  =  1  ...n, 

x,  y  >  0,  xuyi(x )  +  =  0,  i  =  1,  ...n. 

We  then  form  the  Lagrangian  for  this  problem: 

n  n 

C(x,  y,  w,  A,  n)  =  f(x)  +  +  wi)~Y2 

i— 1  *— 1 

n  l  n  n  n 

^  ^  l^yiVi  ^  ^  fJt2i%2i  “1“  ^  ^  ^2 ihi{x)  ^  ^  ^3 i{l/i  9i(x))  ®  ^ 

i=l  i— 1  1  i= 1  i= 1 


(5) 


mm 


s.t. 
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where  (A,  /x)  are  of  course  the  vectors  of  Lagrange  multipliers.  Now  if 
a  given  solution  x*  to  (5)  is  strictly  complementary,  meaning  that  in 
each  complementary  pair  exactly  one  of  and  y*  is  0  (and  the  other 
strictly  positive),  we  may  do  the  following:  Partition  the  set  {1, . . . ,  n} 
into  two  sets  I  and  J,  with  J  representing  those  indices  j  for  which 
x\j  is  0,  and  I  those  i  for  which  y*  is  0.  We  use  the  notation  (standard 
for  interior-point  methods)  that  a  capitalized  variable  name  represents 
a  diagonal  matrix  whose  elements  correspond  to  the  vector  with  the 
same  lower-case  name.  The  KKT  conditions  are  as  follows: 


(6) 


'  VIU/(x*)  ' 

+ 

0 

X\j 

VXlJ/(x*)  J 

.yf . 

'  ti.1 ' 

1 

S7Xuh(x*) 

\* 
a2  I 

S7XlIg(x*) 

\* 
a3  I 

.  Vu  . 

T" 

Vei  jh(x*) 

\* 

L  J 

.  Vxu0( a:*)  _ 

\* 

L  a3j  J 

VyC 

VX2£ 

dC 

dwi 


*  w 

1 _ 

+ 

^31 

'  »*yl  ' 

o 

.  Kj  . 

- 1 

_ 1 

=  0, 


VX2/(z*)  -A  +  VX2Mx*) a;  -  VI25(x*)A;  =  0. 


0  implies  A *uWi  =  6,  i  =  1, . . . ,  n. 


In  addition,  there  are  of  course  primal  feasibility  and  the  usual  com¬ 
plementarity  conditions  for  the  multipliers  fi.  The  algorithm  attempts 
to  get  to  a  solution  to  this  system  with  6  =  0. 

If  x*  is  a  non-strict  complementary  solution,  i.e.  for  some  i,  x*u  = 
yi(x*)  =  0,  then  by  inspection  of  the  KKT  system  we  see  that  the 
corresponding  element  of  A,  drops  out  and  has  no  effect  on  the  system. 
This  results  in  a  rank  deficient  system,  which  can  cause  serious  prob¬ 
lems  for  nonlinear  solvers,  in  particular  for  interior-point  methods  (see 
the  discussion  of  numerical  results  in  Section  6). 

Despite  the  theoretical  difficulties,  unboundedness  of  the  optimal 
multiplier  set  has  thus  far  not  usually  been  a  serious  problem  in  prac¬ 
tice.  Indeed,  a  fair  amount  of  success  has  recently  been  achieved  in  solv¬ 
ing  MPEC’s  as  nonlinear  programs.  LOQO  has  proved  quite  successful 
in  solving  a  large  number  of  MPEC’s  directly  as  nonlinear  programs, 
as  have  SQP  methods  (see  [8]).  This  could  lead  to  the  conclusion  that 
MPEC’s  need  not  be  regarded  as  a  special  problem  class,  and  particu¬ 
lar  methods  for  dealing  with  them  are  largely  unnecessary.  In  Section 
5,  however,  we  present  a  potentially  important  example  which  is  not 
solved  robustly  if  handled  as  an  ordinary  NLP.  Methods  which  bound 
the  optimal  multipliers,  on  the  other  hand,  do  solve  it  successfully. 
These  methods  we  describe  in  the  next  Section. 
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We  conclude  this  Section  with  one  final  observation  which  will  be  im¬ 
portant  later:  assuming  MPEC-LICQ,  it  is  not  hard  to  show  that  the 
KKT  system  (6)  has  Lagrange  multiplier  solutions  for  any  x*  which 
satisfies  the  complementarity  constraints  strictly,  in  other  words  the 
objective  function  /  has  no  role  in  determining  whether  x*  is  a  solu¬ 
tion.  That  is,  every  feasible  point  is  a  KKT  point,  for  any  /,  if  one 
assumes  strict  complementarity.  This  implies  that  the  feasible  set  is 
discrete.  Note  that  this  observation  only  applies  to  problems  in  which 
all  variables  are  involved  in  the  complementarity  constraints. 

4.  Penalty  Methods 
Consider  again  our  NLP  in  the  form 
min  f{x ) 

(7)  s.t.  hi(x)  =  0,i  =  1 . .  .m,  yi  =  gi(x),i  =  l...n, 

x, y  >  0, ,  XuViix)  <  0,  %  —  l...n. 

A  common  approach  in  the  past  to  deal  with  difficult  constraints  in  an 
NLP  has  been  to  replace  them  with  a  penalty  term  in  the  objective 
function,  which  one  tries  to  drive  to  zero.  That  is,  suppose  one  has  the 
problem 

min  fix) 
s.t.  h{x)  =  0. 

This  is  replaced  with  the  penalized  problem 

min  f{x)  +  pP(x). 

Pix)  is  usually  a  non-negative  function  which  equals  0  precisely  when 
hix)  =  0.  The  penalty  parameter  p  is  a  positive  constant.  The  hope 
is  that  making  p  large  enough  will  produce  a  solution  to  the  penalized 
problem  with  P(x)  =  0  (and  hence  a  solution  to  the  original  problem). 
This  is  only  possible  if  a  solution  to  the  original  problem  is  a  solution 
to  the  penalized  problem.  In  fact,  ideally  every  solution  to  the  true 
problem  would  be  a  solution  for  the  penalized  problem.  This  does 
not  have  to  hold  in  general.  The  most  standard  penalty  function,  the 
quadratic  (which  sets  P(x)  =  £\(/ij(x))2)  does  not  have  this  property. 
Penalty  functions  which  do  have  this  property  are  called  exact. 

4.1.  Exact  Penalty  Functions.  The  two  most  common  exact  penalty 
functions  are  the  l\  penalty  function,  defined  by  P(x)  =  |/ij(x)|,  and 

the  4c  defined  by  P(x)  =  maxi{|/ij(x)|}. 

The  exactness  property  guarantees  that  if  one  of  these  penalty  func¬ 
tions  is  used,  then  every  solution  of  the  true  problem  is  a  solution  to  the 
penalized  problem  for  sufficiently  large  (but  finite)  penalty  p.  Thus, 
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if  x *  is  a  solution  to  the  true  problem,  then  picking  a  starting  point 
sufficiently  close  to  x*  when  solving  the  penalized  problem  guarantees 
convergence  to  x*  (i.e.  local  convergence).  This  desirable  property  of 
the  t\  and  the  penalty  functions  is  generally  proved  assuming  con¬ 
straint  qualification  conditions  hold  at  a  solution  to  the  true  problem. 
As  such,  penalty  methods  (i.e.  constraint  relaxation)  have  been  ap¬ 
plied  until  now  to  problems  that  are  assumed  to  be  well-behaved  (see 
e.g.  [10]).  MPEC’s  of  course  are  not  well-behaved,  but  we  shall  see 
that  this  is  not  a  problem. 

First  let  us  see  what  the  t\  and  penalty  functions  look  like  when 
applied  to  the  complementarity  constraints  in  (4): 

For  the  i\  norm  we  have  the  penalized  problem 

min  f(x)  +  p^2\xuyi(x)\ 
s.t.  x,y>0,y  =  g(x),h(x)  =  0, 

which,  because  x  and  g(x)  are  constrained  to  be  non-negative,  is  equiv- 
alent  to 

/  q\  min  f(x)  +pJ2  Xuyi(x) 

''  s.t.  x,y  >  0,2/  =  g{x),h(x)  =  0, 

which  may  also  be  written  as 

min  f(x)  +  pJ20 

(9)  s.t.  xuyi(x)  <  C i,  i  =  1,  •  •  • ,  n, 

x,y>0,y  =  g{x),h(x)  =  0. 

For  the  norm  we  have 

min  f(x)  +  pmaxj  {|a;iiyi(:r)|} 
s.t.  x,y  >  0,y  =  g(x),h(x)  =  0, 

which  can  be  written  as 

min  f(x)  +  pC 

(10)  s.t.  xuyi{x)  <  C,  i  =  1,  •  ■  • ,  n, 

x,y>  0,y  =  g(x),  h(x)  =  0. 

Note  that  (9)  and  (10)  only  involve  differentiable  functions,  so  normal 
optimization  methods  can  be  used. 

It  is  shown  in,  among  other  places,  [1]  that  any  solution  to  (4)  is 
a  solution  to  (10)  for  large  enough  (finite)  p,  in  other  words  the 
penalty  function  is  still  exact,  if  (4)  satisfies  MPEC-LICQ.  What  is 
more,  the  result  in  [1]  implies  that  any  solution  to  (4)  is  a  solution,  for 
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a  large  enough  but  finite  penalty,  to  the  problem 
min  f(x)  +  pC 

n 

(11)  S.t.  y^xnyijx)  <C,i  =  1-n, 

i=  1 

x,y>0,y  =  g(x),h(x )  =  0, 

i.e.  what  one  gets  if  the  original  problem  had  been  written  with  the 
summation  formulation  (see  Section  3).  And  plainly  (11)  is  equivalent 
to  (8).  Hence  the  i\  penalty  function  is  also  exact  when  applied  to 
MPEC’s  (with  MPEC-LICQ). 

Thus  these  penalty  methods  could  be  an  attractive  way  to  solve 
MPEC’s,  if  they  took  care  of  the  original  problem  posed  by  MPEC’s, 
namely  that  the  optimal  set  of  Lagrange  multipliers  is  unbounded.  And 
indeed,  this  is  precisely  what  occurs.  This  is  a  well-known  fact,  and 
is  shown  in,  for  instance,  [1].  It  is  trivial  to  show  that  the  multipliers 
remain  bounded  when  a  LOQO-type  interior-point  method  is  applied  to 
(10)  (or  (9)).  For  completeness  we  now  state  this  as  a  theorem,  since 
it  will  be  crucial  to  all  the  work  that  follows. 

Theorem  1.  If  an  interior-point  method  (as  described  in  section  2)  is 
applied  to  (10),  the  set  of  optimal  Lagrange  multipliers  for  a  strictly 
complementary  solution  is  bounded,  provided  that  (4)  satisfies  MPEC- 
LICQ. 

In  the  case  of  a  solution  to  (10)  being  a  non-strict  complementary 
solution  to  (4),  this  time  there  is  no  difficulty,  the  reason  being  that 
part  of  the  KKT  conditions  to  (10)  is  the  equation: 


n 

(12)  p = y  Aj, 

i=l 

where  the  X's  are  the  mulitpliers  corresponding  to  the  complementarity 
constraints.  (12)  ensures  that  none  of  the  multipliers  will  drop  out  of 
the  system.  One  further  point  to  note  is  that  if  the  original  problem 
does  not  satisfy  MPEC-LICQ,  there  might  not  be  a  KKT  point  (i.e. 
Lagrange  multipliers  might  not  exist).  See  Section  6  for  an  example  of 
what  happens  in  this  case. 

We  now  have  two  penalty  methods,  using  the  t\  and  penalty  func¬ 
tions,  which  are  exact  and  keep  the  optimal  multiplier  set  bounded, 
when  applied  to  MPEC’s  satisfying  standard  constraint  qualifications. 
They  would  therefore  appear  to  be  promising  alternatives  to  solving  an 
MPEC  directly  as  an  NLP.  Some  evidence  for  this  already  exists.  In 
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[17],  the  t\  penalty  method  is  applied  to  pure  complementarity  prob¬ 
lems  (actually  mixed  complementarity  problems),  and  promising  nu¬ 
merical  results  are  obtained.  LOQO  is  a  more  robust  code,  as  it  can 
deal  with  an  indefinite  Hessian  matrix,  as  can  happen  in  non-convex 
problems.  In  such  cases  LOQO  adds  a  diagonal  perturbation  to  the 
Hessian  matrix  H,  making  it  H  +  XI,  where  I  is  the  identity  matrix, 
and  A  is  a  positive  scalar.  A  large  enough  A  makes  H  +  XI  positive 
definite,  and  the  algorithm  may  proceed  successfully  with  this  mod¬ 
ified  KKT  matrix  (see  [20]).  One  thus  expects  that  LOQO  might  be 
quite  successful  at  using  penalty  methods  to  solve  MPEC’s.  To  this 
end,  we  modified  the  interior-point  code  of  LOQO  to  incorporate  com¬ 
plementarity  constraints  and  use  an  penalty  function  to  maintain 
boundedness  of  the  dual  variables.  The  main  thrust  of  this  paper  is 
to  test  the  effectiveness  of  this  implementation.  While  this  paper  does 
not  deal  with  an  l\  penalty  method  for  MPECs,  Section  7  discusses 
a  theoretical  result  that  makes  such  an  implementation  of  interest  for 
general  NLPs.  Future  work  will  report  on  this  implementation. 

5.  An  Example  From  Game  Theory 

We  consider  a  model  that  is  proposed  in  [18]  and  [19]  as  an  exten¬ 
sion  of  the  model  discussed  extensively  in  those  papers  (and  analyzed 
in  [17]).  That  version  of  the  model  is  mentioned  in  [5]  as  being  tradi¬ 
tionally  regarded  as  a  very  hard  equilibrium  problem,  so  we  regard  the 
extended  version  of  the  model  as  a  good  challenge  for  our  efforts.  The 
ampl  model  can  be  found  at  [16]. 

Very  briefly,  the  model  is  of  a  game  involving  two  (identical)  firms 
which  compete  on  the  basis  of  price  to  sell  an  identical  product.  The 
game  takes  place  over  an  infinite  time-horizon  at  discrete  intervals.  In 
each  round,  a  firm  may  or  may  not  be  free  to  move.  If  it  is,  it  has 
the  option  to  choose  a  price  from  a  discrete  set  of  prices  or  sit  out 
of  the  market  for  a  round.  If  it  does  choose  a  price,  it  is  committed 
to  that  price  for  the  following  round  as  well.  The  payoff  matrix  for 
an  individual  round  is  determined  by  a  standard  duopoly  structure 
(see  [18]  and  [19]).  Each  firm  tries  to  maximise  its  future  (discounted) 
payoff.  The  derivation  of  the  necessary  and  sufficient  conditions  for  a 
(mixed-strategy)  Nash  equilibrium  is  similar  to  that  in  [18]  and  [19]  for 
the  simpler  version  of  the  model.  These  can  then  be  formulated  as  an 
MPEC  with  a  constant  objective  function  along  the  lines  of  [17]. 

The  action  space  of  this  problem  is  {-1,0,1 . n  —  1},  where 

0, ...,n  —  1  are  the  n  prices  in  the  model,  and  —1  corresponds  to 
not  chooosing  a  price.  The  payoff  matrix  is  n,  whose  (i,  j)  entry  gives 
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the  payoff  to  a  firm  if  it  chooses  action  i,  and  its  rival  chooses  action 
j.  The  decision  variables  of  this  problem  consist  of: 

i)  A  reaction  matrix  R.  The  (i,  k),i>  0,  entry  of  R  gives  the  prob¬ 
ability  that  the  firm  will  take  action  k  when  its  rival  is  committed  to 
price  i  from  the  previous  round. 

ii)  A  simultaneous-strategy  vector  s.  The  entry  k  of  s  gives  the 
probability  that  the  firm  takes  action  k  when  it  does  not  know  its 
rival’s  price  for  the  current  period. 

iii)  Payoff  functions.  These  are: 

vm  :  the  expected  discounted  payoff  for  a  firm  which  is  free  to  react 
to  its  rival’s  committment  to  price  i  from  the  previous  period 

Vs  :  the  expected  discounted  payoff  for  a  firm  when  both  it  and  its 
rival  are  free  to  choose  new  prices  simultaneously  for  the  current  period 
WRi  :the  expected  discounted  payoff  for  a  firm  when  it  is  committed 
to  price  i  from  the  previous  period,  but  its  rival  is  free  to  react 
wskj  :the  expected  discounted  payoff  for  a  firm  when  it  is  committed 
to  price  k,  and  its  rival  to  price  j  from  the  previous  period 
The  resulting  MPEC  is  then 
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We  attempted  to  solve  the  problem  directly  as  a  nonlinear  program 
using  LOQO,  adopting  the  pairwise  formulation.  The  algorithm  only 
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converged  for  a  few  starting  points  near  an  equilibrium.  Otherwise  it 
failed  for  virtually  all  starting  points.  We  also  tried  the  summation 
formulation;  the  results  were  similar.  With  either  formulation,  LOQO 
usually  reaches  its  iteration  limit  (500) ,  and  the  dual  objective  function 
appears  to  become  unbounded  (more  on  this  later).  To  see  whether 
this  was  a  difficulty  particular  to  LOQO,  we  also  tried  the  problem  on 
some  other  standard  nonlinear  solvers:  KNITRO,  FILTER,  and  snopt. 
KNITRO  is  an  interior-point  code  (though  still  very  different  from  LOQO 
in  its  implementation),  and  FILTER  and  SNOPT  are  SQP  codes  (see 
[9], [10]  and  [4]  for  detailed  descriptions  of  these  solvers).  Here,  too,  the 
results  were  not  satisfactory.  All  the  algorithms  failed  for  a  variety  of 
starting  points  (see  the  appendix  for  the  exact  results). 

Thus  it  appears  that  this  problem  poses  a  genuine  challenge  for 
solvers  when  given  as  a  nonlinear  program.  The  question  is:  Why? 
The  dual  objective  function  reaching  very  high  levels  in  LOQO  is  due 
to  unbounded  duals  going  to  oo.  However,  a  very  large  dual  objective 
function  is  also  often  a  sign  of  an  infeasible  problem.  That  is,  when 
LOQO  has  difficulty  getting  feasible  from  a  given  starting  point  (because 
the  constraints  are  locally  infeasible  in  its  neighborhood),  the  dual  vari¬ 
ables  often  get  very  large.  So  how  can  we  distinguish  between  these  two 
possible  causes?  This  has  always  been  a  difficult  question  to  answer 
(see  [8]).  But  in  this  case  we  have  some  evidence  that  unboundedness 
of  the  multipliers  is  indeed  a  factor. 

Typically,  when  LOQO  has  trouble  getting  feasible  simply  because  the 
constraints  of  a  problem  are  difficult  to  satisfy,  the  algorithm  hardly 
moves  from  the  starting  point.  On  the  other  hand,  if  unbounded  duals 
are  the  source  of  all  the  difficulties,  some  progress  might  be  possible 
before  the  step  lengths  get  too  small.  So  the  following  result  is  sugges¬ 
tive: 

For  the  starting  point  which  sets  all  the  payoffs  to  63  and  all  the 
probabilities  to  2  (with  starting  slacks  at  8.1),  LOQO,  as  usual,  runs 
out  of  iterations  and  the  dual  objective  increases  without  bound.  After 
500  iterations  the  point  LOQO  has  reached  is 
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From  [18]  and  [19]  it  is  clear  that  this  point  appears  to  be  agoniz¬ 
ingly  close  to  an  equilibrium,  indicating  that  unbounded  multipliers 
may  have  prematurely  stalled  progress  of  the  algorithm.  The  following 
example  is  also  interesting: 

We  experimented  with  a  very  simple,  but  related  model.  In  this 
model,  the  firms  only  move  simultaneously,  and  each  time  they  must 
choose  a  price  (no  option  to  wait).  All  other  rules  remain  the  same. 
The  equations  for  the  MPEC  are: 

sk[vs  -  22  s'(n«  +  *>$)]  =  0  V  prices  k, 
i 

vs  ~  22  +  Svs)  ^  0,  sk  >  0  V  prices  k, 

S>  =  i- 

k 

This  model  has  exactly  two  equilibria,  if  we  keep  the  same  payoff 
structure  and  discount  factor  as  before:  both  firms  set  the  price  to  1 
with  probability  1,  or  to  0  with  probability  1  (and  associated  payoffs 
are  25  and  0  respectively).  Even  on  this  problem  LOQO  fails  from  a  few 
starting  points  (though  not  from  as  many  as  for  the  full  model).  One 
such  starting  point  sets  the  payoff  variable  to  70  and  all  probabilities 
to  .14  (and  starting  slacks  set  to  .0001).  With  this  starting  point, 
as  before  LOQO  runs  out  of  iterations  and  the  dual  objective  function 
grows  without  bound.  After  500  iterations  the  point  LOQO  has  reached 
is 

payoff  =  23.03,  s  =(  .09  .52  -1.77e~17  .03  .13  .09  .1  ). 

It  is  immediately  clear  that  LOQO  was  moving  towards  one  of  the 
equilibria  (price  =  1,  payoff  =  25),  when  it  stalled.  This  is  even  more 
interesting  than  the  previous  example.  LOQO  can  sometimes  have  nu¬ 
merical  difficulties  near  a  solution.  In  this  case  it  stalled  near  neither 
a  solution  nor  the  starting  point,  but  somewhere  in  between,  while 
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apparently  making  good  progress.  This  argues  still  more  strongly  in 
favor  of  unbounded  duals  causing  difficulties  (though  this  is  not  to 
say  that  there  aren’t  other  complicating  factors  involved,  which  are 
demonstrated  below). 

We  also  looked  at  the  simpler  model  which  is  analysed  in  [17],  [18] 
and  [19].  In  this  model  the  firms  always  alternate  their  moves,  and  here 
too  there  is  no  option  to  wait.  Thus  each  round  one  firm  is  moving,  and 
the  other  is  committed  to  its  price  from  the  previous  round.  We  ran 
this  model  on  LOQO,  and  failure  was  still  common  for  many  starting 
points.  This  time,  the  summation  formulation  was  clearly  better  than 
the  pairwise,  though  it  too  still  failed  for  various  starting  points  (the 
pairwise  failed  almost  everywhere).  This  is  not  surprising  behavior, 
as  the  summation  formulation  can  indeed  be  better  in  the  presence  of 
unbounded  multipliers,  as  then  there  are  fewer  multipliers  that  can  go 
to  oo  (and  hence  fewer  slack  variables  which  become  small). 


5.1.  Solving  the  problem  using  penalty  methods.  The  t\  and 
penalty  methods  worked  very  well  on  all  three  models  discussed  (the 
main  model,  as  well  as  the  alternating-move  and  simultaneous-move 
models).  Both  methods  work  for  almost  all  starting  points,  including 
starting  slack  values.  They  fail  on  only  two  (rare)  occasions: 

1.  If  the  starting  slack  value  is  very  small  (as  mentioned  before, 
LOQO  can  stall  if  slack  variables  get  too  close  to  0). 

2.  Prom  some  starting  points,  the  algorithm  returns  what  might  be 
called  a  “false”  solution,  i.e.  a  solution  to  the  penalized  problem  which 
is  infeasible  for  the  true  problem.  Meaning  in  this  case,  all  the  con¬ 
straints  are  satisfied  except  that  the  complementarity  is  positive.  We 
knew  this  could  happen,  since  these  methods  are  not  globally  conver¬ 
gent.  Still,  this  happens  rarely,  and  changing  the  starting  point  just 
slightly  generally  fixes  the  problem. 

Here  are  two  examples  of  the  equilibria  we  found  using  the  penalty 
method: 

Setting  starting  probabilities  to  .2  and  starting  payoffs  to  30,  LOQO 
reached  the  following  equilibrium  after  57  itertions: 
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s  =  (  0  .83878  0  0  0  0  0  .16122  ) , 

’  0  0  1  0  0  0  0  0  ' 

0  .619308  .380692  00000 
0  0  1  0  0  0  0  0 

R  =  0  0  1  00000. 

0  0  1  0  0  0  0  0 

0  0  1  0  0  0  0  0 

.  0  0  1  0  0  0  0  0  . 

Setting  starting  probabilities  to  .8  and  payoffs  to  63,  LOQO  reached 
the  following  equilibrium  after  60  iterations: 

a  =  (01000000  ), 

'  .360441  0  0  0  0  .152644  .379981  .106935  ' 

0  1  0  0  0  0  0  0 

0  1  0  0  0  0  0  0 

R  =  001000  0  0 

0  0  0  1  0  0  0  0 

0  0  0  1  0  0  0  0 

0  0  0  1  0  0  0  0 

To  try  to  improve  convergence  still  further,  we  tried  to  modify  the 
penalty  problem  (10)  by  adding  an  upper  bound  to  the  infeasibility 
of  the  complementarity  constraint,  (,  and  gradually  tightening  it  as 
needed.  For  a  few  starting  points  from  which  we  had  been  getting  to 
a  “false”  optimum  before,  now  we  got  to  a  true  solution.  But  perfor¬ 
mance  was  actually  worse  for  most  starting  points.  In  fact,  setting  the 
bound  on  £  to  .05  resulted  in  failure  from  almost  all  starting  points. 
In  practice  it  appears  best  to  leave  the  infeasibility  unbounded  above, 
and  shift  the  starting  point,  if  necessary,  to  get  a  true  solution  to  the 
problem. 

The  findings  just  described  are  crucial.  We  have  an  example  of  a 
problem  with  bounded  multipliers  at  the  solutions  (and  which  hence 
obeys  constraint  qualifications)  on  which,  nonetheless,  LOQO  fails  from 
almost  everywhere.  The  only  explanation  for  this  is  the  extreme  non¬ 
convexity  of  the  problem,  which  is  due  to  the  problem  having  a  large 
set  of  discrete  equilibria.  The  discreteness  arises  from  the  strict  com¬ 
plementarity  of  the  solutions,  which  is  typical  for  a  game  theoretic 
problem,  as  at  equilibrium  actions  are  rarely  played  with  probability 
0  if  they  are  as  good  as  the  actions  actually  being  played.  Such  a 
non-convex  problem  can  have  many  locally-infeasible  points. 
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Definition  1.  A  point  x  is  locally  infeasible  if  hi(x)  ^  0  for  some 
i,  and  x  is  a  local  minimizer  of  the  function  Yli\^i(x) |  (assuming  a 
problem  with  feasible  set  hi(x)  =  0,  i  =  1, . . . ,  m). 

Algorithms  commonly  converge  to  locally  infeasible  points,  and  to 
confirm  that  many  locally  feasible  points  were  present,  we  also  at¬ 
tempted  to  solve  the  problem  by  relaxing  all  the  constraints  (i.e.  not 
just  the  complementarity  constraints).  Indeed,  from  most  starting 
points  we  got  non-zero  solutions.  This  suggests  that  unbounded  multi¬ 
pliers  are  not  the  only  factor,  or  perhaps  even  the  main  factor,  making 
our  problem  difficult.  This  is  not  very  surprising,  as  unbounded  mul¬ 
tipliers  can  be  expected  to  cause  trouble  only  if  a  problem  is  already 
difficult  for  some  reason.  We  know  that  penalty  methods  bound  the 
multipliers,  and  that  they  can  remove  this  extreme  non-convexity  may 
be  explained  as  follows: 

Our  problem  has  the  form 

min  0 

/,  n\  S.t.  Xigi{x)=  0,  i  =  l,...,n, 

'  x  >  0,  g(x )  >  0, 

hj(x)  =  0,  j  = 

If  we  consider  the  method,  the  penalized  problem  in  this  case 
looks  like: 

c 

Xigi{x)<  C,  i  =  1, . . .  ,n, 

C  >  0,x  >  0 ,g(x)  >  0, 
hj(x)  =  0,  j  = 

Assuming  g  and  h  are  well  behaved,  we  expect  (14)  to  have  a  contin¬ 
uous  feasible  set,  indeed  to  obey  standard  constraint  qualifications.  In 
other  words,  (14)  differs  from  (13)  in  that  it  removes  the  complicating 
constraint  from  (13).  Thus  the  constraints  in  (14)  can  be  expected  to 
produce  far  fewer  locally  infeasible  points  than  those  in  (13).  (14)  can 
of  course  produce  “false”  optima  as  described  above,  but  these  can  only 
occur  over  the  feasible  set.  That  is,  the  region  in  which  the  algorithm 
can  fail  to  converge  to  a  true  solution  is  much  smaller  for  (14)  than 
(13).  (14)  also  bounds  the  multipliers,  of  course,  so  we  believe  this 
is  a  comprehensive  explanation  for  why  the  penalty  method  is  greatly 
superior  to  the  direct  approach  on  this  problem.  As  there  can  be  many 
kinds  of  equilibrium  problems  which  produce  large,  discrete  feasible 
sets  (game  theory  being  only  one  example),  this  indicates  that  penalty 
methods  could  be  indispensable  for  solving  MPEC’s  in  general. 
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6.  Other  MPEC’s 

We  also  tested  the  efficacy  of  penalty  methods  on  the  MacMPEC 
test  set  found  in  ([21]).  As  these  problems  are  mostly  true  MPEC’s 
(i.e.  with  an  objective  function),  two  major  issues  were  how  to  choose 
the  initial  value  of  the  penalty  parameter,  and  how  to  update  it  during 
the  course  of  the  algorithm.  Using  LOQO,  we  chose  to  implement  the 
ioo  penalty  method  to  relax  the  complementarity  constraints.  The  first 
step  was  to  initialize  the  penalty  parameter. 

Initialization  was  done  as  follows:  Given  an  initial  value  A0  for  the 
Lagrange  multiplier  of  the  bound  constraint  on  the  auxiliary  variable 
C,  and  initial  values  Ai, . . . ,  Am  for  all  the  other  constraints,  we  set  the 
initial  value  of  the  parameter  p  to  be  simply 

m 

p = aq + Aj. 

i= 1 

The  initial  Lagrange  multiplier  estimates  are  obtained  by  solving  the 
system 

-H  +  I  AT  1  [  x  1  _  [  V / (x0) 

A  I  A  [  h(x°) 
where  H,A,h  are  as  defined  in  Section  2.  The  system  above  is  de¬ 
rived  from  the  KKT  system  shown  in  Section  2,  with  6  set  to  0.  We 
emphasize  that  all  the  multiplier  estimates  (not  just  those  for  the  com¬ 
plementarity  constraints)  are  used  in  the  calculation  of  the  initial  p. 
This  is  done  to  get  a  better  estimate  of  the  order  of  magnitude  of  the 
numbers  in  the  problem. 

The  perturbation  variable  is  initialized  at  the  value  of  the  LOQO 
option  bndpush,  which  is  also  the  initial  value  of  all  the  slack  variables. 
The  default  value  of  bndpush  is  1. 

The  updating  of  the  penalty  parameter  occurs  whenever  the  algo¬ 
rithm  is  approaching  optimality  with  a  scaled  perturbation,  C,  that  is 
greater  than  10"6  .  The  scaling  of  the  perturbation  is  done  as  follows: 

C  = _ £ _ 

maxi(l,xi,5i(x)) 

The  scaled  perturbation  is  used  as  a  better  measure  of  what  constitutes 
“0”  given  the  magnitude  of  the  numbers  in  the  problem. 

When  C  >  10“6  at  the  conclusion  of  the  algorithm,  the  penalty 
parameter  and  the  dual  variable  associated  with  the  bound  on  the 
perturbation  are  updated: 

pk+1  =  l0p\  Xk+l  =  Xk  +  pk+1  -  pk 
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After  the  update,  the  algorithm  continues  to  make  progress  from  the 
previous  values  for  all  other  variables,  thereby  performing  a  “warm 
start” .  Updating  the  dual  variable  as  given  ensures  that  dual  feasibility 
will  be  maintained. 

Note  that  at  present  we  do  not  have  a  way  to  deal  with  the  situation 
where  LOQO  fails  to  find  a  solution  at  all  (i.e.  even  a  ’’false”  optimum) 
for  too  small  a  value  of  the  penalty  parameter.  An  example  of  this 
behavior  is  mentioned  in  the  numerical  results  below.  It  will  require 
further  investigation. 

6.1.  Numerical  Results.  We  tried  our  algorithm  on  two  test-sets: 
the  MacMPEC  test-set  already  mentioned  (found  at  [21]),  and  a  test- 
set,  described  in  [5],  of  pure  complementarity  problems  which  have 
usually  been  regarded  as  difficult  for  complementarity  solvers.  We 
were  able  to  solve  almost  all  the  problems  successfully.  Moreover, 
our  technique  for  estimating  the  initial  value  of  the  penalty  param¬ 
eter  proved  quite  effective.  The  penalty  method  was  able  to  solve  some 
problems  which  a  standard  implementation  of  LOQO  was  unable  to 
solve.  These  problems  were  ex9.2.2,  pack-rig2-16,  pack-rig2p-32,  qpec2, 
ralphl,  scholtes4,  tap-09,  and  tap-15. 

The  full  results  are  given  in  the  appendix.  We  give  not  only  the 
results  for  LOQO,  but  for  sake  of  comparison,  the  results  for  a  standard 
SQP  solver,  FILTER  (see  [9]  for  details  on  filter).  As  reported  in 
[8],  FILTER  also  appears  to  be  quite  successful,  particularly  on  the 
MacMPEC  set.  Now  we  mention  those  few  problems  which  did  cause 
some  difficulties  for  LOQO.  There  were  seven  such  problems  in  the 
MacMPEC  set,  labeled  as  ex9.2.2,  ralphl,  scholtes4,  ralph2,  siouxflsl, 
water-net  and  water-FL. 

ralph2  is  the  problem 

min  x2  —  y2  —  4  xy 
s.t.  xy  =  0, 

x  >0,y  >0. 

The  too  penalty  method  applied  to  this  problem  results  in  the  penalized 
problem 

min  x2  —  y2  —  ixy  +  p( 

s.t.  xy  <  C, 

x  >  0,  y  >  0, 

which  may  be  rewritten  as 

min  (x  -  y)2  -  2 xy  + 
s.t.  xy  <  (, 

x  >  0,y  >  0. 
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It  can  be  seen  by  inspection  that  for  p  <  2,  making  x  and  y  arbitrarily 
large  while  keeping  x  =  y,  and  keeping  (  =  xy,  makes  the  objective 
function  arbirtrarily  small  while  staying  within  the  feasible  set.  In  other 
words,  the  penalized  problem  is  unbounded  for  too  small  a  value  of  the 
penalty  parameter.  As  a  result,  LOQO  might  not  return  a  solution, 
and  our  method  as  outlined  would  stall.  In  this  case  increasing  p  to 
be  at  least  2  resolves  the  issue.  But  as  yet  we  have  not  found  a  way 
to  handle  the  situation  in  general  in  which  too  small  a  penalty  might 
produce  an  unbounded  (or  otherwise  badly-behaved)  problem,  not  to 
mention  cases  where  the  penalized  problem  can  remain  badly-behaved 
for  any  finite  value  of  the  penalty.  This  is  all  the  subject  of  future 
research. 

ralphl  is  the  problem 

min  2  x  —  y 
s.t.  y(y  -  x)  =  0, 

y  >  0,  y  —  x  >  0,  x  >  0. 

The  KKT  conditions  are 

2  —  Xy  +  pi  —  P3  =  0) 

—1  +  2\y  —  \x  —  pi  —  p,2  =  0, 

(plus  the  complementarity  and  feasibility  conditions),  where  A  is  the 
multiplier  for  the  complementarity  constraint,  and  p\  corresponds  to 
y  —  x  >  0,  p2  to  y  >  0,and  p^to  x>  0. 

It  is  clear  by  inspection  that  the  point  (0, 0)  is  the  unique  solution  to 
this  problem.  But  substituting  (0, 0)  for  (x,  y)  in  the  second  equation 
above  yields 

—  1  =  Pi  +  P2 

where  p\  and  p2  are  nonnegative  multipliers  as  they  correspond  to 
inequality  constraints.  Thus  the  KKT  system  has  no  solution  at  the 
point  (0,0). 

The  penalized  version  of  the  above  problem  is 

min  2x  —  y  +  p( 
s.t.  y(y  -  x)  <  C, 

(>  0,y  >  0,y  —  x  >  0,  x  >  0. 

The  KKT  conditions  are 

2  -  Ay  +  pi  -  p3  -  0, 

-l  +  2\y- Xx- pi- p2  =  0, 

p  —  A  +  Pi, 

(plus  complementarity  and  feasibility  conditions). 
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Again  it  is  the  case  that  at  the  point  (0, 0)  there  is  no  nonnegative 
solution  for  for  any  value  of  the  penalty.  Hence  any  nonlinear 

solver  which  tries  to  find  KKT  points  will  run  into  trouble,  and  as  noted 
in  the  appendix,  LOQO  could  not  find  the  solution  to  this  problem. 
Interestingly,  what  it  did  find  was  the  point  (0,  ^),  with  C  =  4^ »  for 
a  fixed  p,  which  does  admit  a  KKT  solution.  As  p  gets  large  this 
solution  will  asymptotically  converge  to  the  non— KKT  point  (0,0).  In 
other  words,  the  usually  exact  l0 o  penalty  function  starts  to  resemble 
an  inexact  function  (exactness  of  course  cannot  hold  when  no  KKT 
solution  exits).  There  is  reason  to  think  that  this  kind  of  behavior  may 
be  observed  in  general: 

For  a  penalized  problem 

min  f{x)  +  pP{x) 

where  P{x)  =  0  represents  a  feasible  point,  as  p  gets  very  large,  solu¬ 
tions  to  the  above  problem  would  be  expected  to  converge  to  a  point 
which  locally  minimizes  P{x).  Thus  for  a  good  enough  starting  point 
convergence  to  the  solution  to  the  original  problem  can  be  achieved, 
even  if  that  solution  is  not  a  KKT  point.  Moreover,  this  would  ap¬ 
ply  to  problems  in  general  with  non-KKT  points  as  solutions,  not  just 
MPEC’s.  In  fact,  we  have  observed  the  same  behavior  for  problem  13 
of  the  Hock  and  Schittkowski  test  set  [11].  We  now  prove  this  quickly 
under  fairly  general  conditions. 

Theorem  2.  Given  the  following  nonlinear  programming  problem: 
min  f(x) 

s.t.  hi(x)  >0,*  =  l,...,m. 

Let  f(x)  and  hi(x )  for  all  i  be  twice  continuously  differentiable.  Let  x* 
be  a  solution  to  the  above  problem.  Suppose  that  x*  is  the  unique  solu¬ 
tion  in  a  neighborhood  ofBr(x*),  r  >  0.  Let  there  be  given  a  monotonic 
sequence  0  <  p\  <  . .  •  <  Pk  such  that  lim^oc  Pk  —  00  and  let  (2^,  Cfc) 
be  the  unique  solution  with  x  6  Br(x*)  to  the  problem 

min  f{x)  +  pkC 

(15)  s.t.  hi(x)  >  — =  1, . . .  ,m, 

C>o. 

Then  lim*— <»  Xk  —  x*. 

Proof.  By  assumption,  we  have  for  all  k, 

f{xk)  +  Pk^k  <  f(xk+l)  +  PkCk+l, 
f{xk+ 1)  +  Pk+lCk+1  5:  f{xk)  +  Pk+lCk- 
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Therefore, 

(Pk+l  -  Pk)Ck  >  ( Pk+l  -  Pk)Ck+l  Vfc, 
which  implies  that 

(k+i  <  Ck  VA;. 

In  other  words,  {(&}  is  a  monotonically  decreasing  sequence  bounded 
from  below,  hence  it  converges  to  some  The  first  inequality  above 
also  gives  us 

f(xk+i)  -  f(xk)  >  pk(Q  -  (k+ 1)  >  0  Vfc. 

Thus  {f(xk)}  is  an  increasing  sequence.  Since  (x*,0)  is  feasible  for 

(15) ,  for  any  pk,  we  have 

(16)  f(xk)  +  pkCk  <  f(x*)  Vfe, 

which  implies  {f(xk)}  is  bounded  above.  Hence  {f(xk)}  converges  to 
some  {/(£)}>  and  by  the  continuity  of  /,  {xk}  converges  to  x .  We  now 
only  need  to  show  that  £  =  0: 

(16)  implies 

f(x *)  —  f(Xk) 

4  <  Vfc. 

PA: 

This  gives 

c=  lim  Cb  <  lim  /(l’>  - /(lt)  <  lim  !(x’]  - M  =  Q. 

fc”+oo  k-^oo  pk  k—>oo  pk 

Since  C  =  0,  it  follows  that  x  must  be  feasible.  Since  {f{xk}}  is  bounded 
above  by  we  have  f(x)  <  f(x*).  The  last  equality  follows  from 

the  fact  that  pk  — *  oo  as  k  — »  oo.  Therefore  x  =  x*,  since  x*  is  by 
assumption  the  only  solution  to  our  problem.  □ 

The  crucial  point  to  note  is  that  no  assumption  is  made  about  con¬ 
straint  qualifications.  All  that  is  required  for  this  theorem  to  be  ap¬ 
plicable  in  some  neighborhood  of  a  solution  is  that  the  sequence  of 
penalized  problems  have  KKT  solutions.  This  can  be  ensured  if  the 
constraint  gradient  matrix  for  the  penalized  problems  is  nonsingular, 
and  this  can  be  guaranteed  for  the  t\  penalty  function  (see  the  next 
Section). 

The  problems  ex9.2.2  and  scholtes4,  like  ralphl  also  do  not  have 
KKT  points  and  similarly  allow  for  only  approximate  answers.  We 
note  that  these  three  problems  have  non-strict  solutions  (the  extra  lin¬ 
ear  dependence  is  what  keeps  multipliers  from  existing),  but  the  non¬ 
strictness  as  such  did  not  cause  any  problems  for  the  penalty  method. 
Indeed,  there  are  over  50  problems  in  the  MacMPEC  set  which  have 
at  least  one  non-strict  pair,  and  with  the  exception  of  the  ones  just 
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described,  the  penalty  method  solved  them  all.  This  is  in  contrast  to 
what  happens  when  LOQO  is  used  to  solve  the  problems  directly,  with¬ 
out  relaxation.  In  that  case  three  non-strict  problems  ( tap-09,  tap-15, 
and  qpec2),  which  do  have  KKT  solutions,  caused  difficulties.  These 
are  examples  of  rank-deficiency  (which  occurs  in  non-strict  problems, 
as  shown  in  Section  3)  hurting  an  interior-point  method  for  numerical 
reasons.  The  penalty  method  avoids  the  issue  of  rank  deficiency  al¬ 
together  by  keeping  all  multipliers  within  the  system  (and  bounded). 
The  problems  siouxflsl ,water-FL  and  water-net  are  rank-deficient  even 
after  relaxation,  i.e.  the  non-complementarity  constraints  are  not  well- 
behaved.  Hence  trouble  with  these  problems  is  not  surprising. 

The  other  test  set  (of  pure  complementarity  problems)  consisted  of 
11  models.  Of  these  we  solved  8,  with  minor  adjustments  as  needed  (de¬ 
scribed  in  the  appendix).  One  model  had  serious  errors  which  caused 
the  AMPL  processor  to  fail,  and  the  two  problems  on  which  genuine  fail¬ 
ure  occurred  had  functions  which  were  not  differentiable  at  some  point. 
This  can  cause  serious  numerical  difficulties  for  any  method  based  on 
the  assumption  of  differentiability. 

Thus  the  numerical  evidence  shows  that  penalty  methods,  in  con¬ 
junction  with  an  interior-point  solver,  can  successfully  resolve  the  diffi¬ 
culties  posed  by  equilibrium  constraints.  In  one  framework  we  are  able 
to  deal  with  strict  and  non-strict  problems,  and  even  problems  without 
KKT  solutions.  Moreover  our  implementation  does  not  even  require 
that  the  multipliers  at  the  solution  be  unique,  merely  bounded.  This 
is  due,  we  surmise,  to  the  property  that  interior-point  methods  have 
of  going  to  the  center  of  the  face  of  solutions.  And  we  have  some  nu¬ 
merical  evidence  for  this.  Consider,  for  example,  the  problem  tap-15: 
the  optimal  value  of  the  penalty  parameter  was  1000.  There  were  83 
complementarity  constraints,  and  their  optimal  multipliers  summed  to 
994,  with  all  of  them  having  values  between  9  and  12,  and  most  of 
them  being  almost  equal.  In  other  words,  the  algorithm  really  did  go 
to  the  center  of  the  face  of  optimal  solutions. 

7.  Penalty  Methods  for  Nonlinear  Programming 

In  the  previous  section,  we  showed  that  penalty  methods  can  aid  in 
finding  the  optimal  solution  when  no  KKT  point  exists.  It  can  be  seen 
from  the  proof  of  Theorem  3  that  this  property  applies  in  general  to 
NLPs.  In  this  section,  we  outline  two  more  issues  that  can  be  resolved 
using  penalty  methods  in  the  context  of  general  NLP. 

7.1.  Jamming  in  Interior-Point  Methods.  A  problem  that  interior- 
point  methods  such  as  LOQO  have  experienced  in  the  past  is  that  of 
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jamming.  Jamming  occurs  when  variables  that  must  be  kept  non¬ 
negative  (in  loqo’s  case,  the  slack  variables)  attain  a  value  of  0  and 
the  descent  direction  followed  by  the  algorithm  in  the  next  iteration 
would  make  them  negative.  In  this  situation  an  interior-point  method 
is  forced  to  cut  its  steplength  to  0  and  the  algorithm  stalls.  In  [3]  it  is 
shown  that  jamming  will  not  occur  in  LOQO  for  non-degenerate  prob¬ 
lems  (i.e.  problems  with  strict  complementarity  throughout  between 
slack  and  dual  variables)  if  the  constraint  gradient  matrix  for  a  partic¬ 
ular  subset  of  the  constraints  is  nonsingular.  We  now  show  that  the 
use  of  the  penalty  method  to  relax  all  the  constraints  of  a  problem 
will  ensure  that  this  is  always  the  case. 

(In  what  follows,  once  again  as  is  standard  with  the  notation  of 
interior-point  methods,  a  capitalized  variable  name  represents  a  diag¬ 
onal  matrix  whose  elements  are  the  coordinates  of  the  corresponding 
vector  variable;  Z  will  be  used  for  a  capital  (.) 

For  simplicity  of  notation  we  will  consider  problems  of  the  form 

/17\  min  f(x) 

v  '  s.t.  hi(x)  >  0,  i  =  l,...,m. 

Theorem  3.  Assuming  (17)  is  non- degenerate  as  defined  above,  LOQO 
will  not  jam  in  the  sense  of  [3]  if  the  i\  penalty  method  is  applied. 


Proof.  We  relax  each  constraint  with  a  separate  auxiliary  variable  to 
obtain  the  penalized  problem 


(18) 


min 

s.t. 


f(x)+p^2ci 

2—1 


C»  >  0,9i(x)  >  -Ci, 


i  =  1, . . .  ,m. 


The  auxiliary  variables  are  then  treated  like  slack  variables  and  put  in 
the  barrier  function: 


(19) 


min 

s.t. 


f(x) + p  e*  -  Qi  5^  log  c< 


2=1 


2=1 


9i(x)  +  Ci>  0,  i  =  1, . . .  ,m. 


The  constraint  gradient  matrix  of  (19)  is 

/ 

Vpi(x)  .  .  .  Wgm{x ) 
which  will  always  be  nonsingular. 
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We  then  add  a  slack  variable  to  each  of  the  original  constraints  and 
put  them  in  the  barrier  function  as  usual: 

mm  m 

min  f(x)  +  pT 

(20)  i=i  i=i  »=i 

s.t.  gi{x)  -  Wi  +  Q  —  0,  i  = 

The  Lagrangian  for  the  above  problem  is 

m  m  m 

C(x,w,t,  A)  =  /(x)+p£G-0iE1°S^-02EloSu;~AT(5(a;)_u;+C)- 


i=i  i=i  *=i 

The  KKT  conditions  are 

V/(x)  -  g{x)TX  =  0, 
pe-QxZ-xe-\  =  0, 

-92W~1e  +  A  =  0, 
g(x )  -  w  +  C  =  0. 


The  second  equation  of  the  above  system  may  be  rewritten  as 

pZe  —  Z\  —  d^e, 

which  in  turn  is  equivalent  to 

(pi  -  A )Ze  =  die  (assuming  C  >  0). 

Similarly  the  third  equation  is  equivalent  to 

W  he  =  d2e. 

We  note  that  the  second  condition  implies  that  the  auxiliary  variables 
£  be  nonnegative  at  all  times,  and  that  the  boundedness  of  the  dual 
variables  A  guarantees  that  there  exists  a  finite  p  such  that  (pi  -  A)  is 
strictly  positive.  In  that  case,  the  system  above  will  converge  to  the 
true  system  (for  (17))  as  9U  d2  — »  0 if  C  — >  Oas  well.  And,  Newton  s 
method  applied  to  this  system  will  not  jam.  This  conclusion  is  valid 
because  the  KKT  system  for  (20)  is  the  system  that  is  obtained  for  the 
barrier  problem  applied  to  (19)  for  any  particular  value  of  6V  (19)  is  an 
optimization  problem  with,  as  already  noted,  a  nonsingular  constraint 
gradient  matrix  at  all  times.  This  along  with  the  nondegeneracy  as¬ 
sumption  meets  the  requirements  of  the  anti-jamming  theorem  in  [3]. 
Finally,  it  can  be  seen  that  the  nonnegative  variables  C  will  not  ’’jam” 
either  against  their  boundaries: 

The  Newton  system  for  the  KKT  system  to  (20)  will  produce  the 
equation 

(pi  -  A)AC  -  CAA  =  dxe, 
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which  implies  that  if  £  =  0,  >  0.  □ 

Note  that  the  above  conclusion  will  not  hold  if  the  lx.  penalty  method 
is  used.  This  is  an  important  theoretical  distinction  between  the  two 
exact  penalty  methods. 

7.2.  Infeasibility  Detection.  A  further  use  of  penalty  methods  is  in 
infeasibility  detection,  meaning  the  identification  of  a  point  as  locally 
infeasible.  For  this  purpose,  of  course,  all  constraints  in  a  problem  need 
to  be  relaxed.  We  did  so  for  one  of  the  problems  in  the  MacMPEC  test 
set  which  is  known  to  be  infeasible,  pack-rig2c-16,  using  the  loo  penalty 
function.  We  obtained  the  following  results: 

penalty  optimal  perturbation 

10  1.102629E-01 

100  1.270534E-01 

1000  1.026225E-01 

10000  1.026297E-01 

Not  unexpectedly,  the  algorithm  reaches  a  point  where  it  is  unable 
to  improve  feasibility  any  further,  regardless  of  how  large  it  makes  the 
penalty.  Thus  relaxing  all  constraints  provides  a  way  to  identify  local 
infeasibility  as  the  cause  for  an  algorithm  to  stall.  And  we  plan  to 
implement  this  as  an  “elastic  mode”  for  LOQO  in  the  future,  to  do 
exactly  that. 


8.  Conclusion 

In  this  paper  we  discussed  the  question  of  solving  an  MPEC  as  a 
nonlinear  program.  This  poses  theoretical  difficulties,  both  in  terms  of 
unbounded  Lagrange  multipliers  at  a  solution,  and  often,  a  discrete  fea¬ 
sible  set  resulting  in  extreme  non-convexity.  We  gave  an  example  where 
these  difficulties  have  a  real  effect  and  showed  that  penalty  methods 
can  resolve  these  difficulties  in  theory,  and  do  so  in  practice  for  our  ex¬ 
ample.  We  went  on  to  implement  a  version  of  the  penalty  method  with 
the  interior-point  code  LOQO,  which  successfully  solved  almost  all  the 
problems  in  two  standard  test-sets.  Furthermore,  our  penalty  method 
solved  some  problems  elegantly  which  before  had  required  an  ad-hoc 
approach.  The  clear  conclusion  is  that  penalty  methods  appear  to  be  a 
good  way  to  solve  MPEC’s  and  equilibrium  problems  in  general  using 
nonlinear  solvers.  We  have  even  shown  that  the  l\  penalty  method  can 
actually  resolve  a  general  difficulty  that  interior-point  methods  such 
as  LOQO  have  experienced  in  the  past.  Thus  the  utility  of  penalty 
methods  potentially  extends  far  beyond  equilibrium  problems. 
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With  regards  to  equilibrium  problems,  a  robust  tool  for  computing 
equilibria  can  render  difficult  but  important  questions  tractable,  and 
open  up  a  vast  number  of  areas  for  application.  In  particular,  the  sta¬ 
bility  of  equilibria  and  the  dynamics  of  systems  in  the  vicinity  of  equi¬ 
libria  can  be  studied  more  successfully  if  the  equilibria  can  actually  be 
found.  In  terms  of  applications,  game  theory  is  both  a  source  of  some 
of  the  hardest  equilibrium  problems  (such  as  the  one  presented  here), 
and  a  subject  of  intense  scrutiny  at  present  because  of  its  applicability 
to  many  important  areas.  The  energy  sector  is  one  such  area,  while  an¬ 
other  is  auctions.  Auctions  are  becoming  increasingly  important,  both 
in  public  sector  activities,  e.g.  spectrum  auctions  for  wireless  com¬ 
panies,  and  in  the  private  sector,  for  instance  in  business-to-business 
e-commerce.  Thus  there  should  be  an  abundance  of  important,  realistic 
problems  to  consider. 

8.1.  Acknowledgement.  We  would  like  to  thank  Sven  Leyffer  for 
providing  us  with  the  FILTER  code  for  use  in  comparative  testing. 
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Appendix  A.  Performance  op  other  nonlinear  codes  on 

THE  MAIN  (GAME  THEORY)  MODEL 

We  tested  the  main  model  on  some  of  the  nonlinear  codes  on  the 
NEOS  server  (www-neos.mcs.anl.gov/neos/).  A  naive  type  of  starting 
point  was  used  in  which  all  the  probabilities  are  set  to  the  same  number 
(between  0  and  1)  and  all  the  payoff  variables  are  set  to  the  same 
number.  The  results  were  as  follows: 


PAIRWISE  FORMULATION 

start  pt. 

solver 

result 

reason  given  (if  any) 

.8/63 

fail 

infeasible  problem 

.8/63 

optimal 

KNITRO 

optimal 

BSflHI 

SNOPT 

fail 

no  further  improvement  possible 

.8/80 

FILTER 

fail 

locally  infeasible 

.8/80 

KNITRO 

optimal 

.2/20 

SNOPT 

optimal 

.2/20 

FILTER 

fail 

locally  infeasible 

.2/20 

KNITRO 

fail 

iteration  limit 

SUMMATION  FORMULATION 

start  pt. 

solver 

result 

reason  given  (if  any) 

SNOPT 

fail 

infeasible  problem 

EiMJMi 

FILTER 

fail 

locally  infeasible 

Esim 

iteration  limit 

SNOPT 

optimal 

FILTER 

optimal 

SEHE 

KNITRO 

optimal 
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Appendix  B.  Performance  of  penalty  methods  on  other 

MPEC’s 

For  the  numerical  testing,  the  two  solvers  used  were:  LOQO  (Version 
6.04)  (see  [20]  for  details)  and  filterMPEC  (Version  20020621)  (see  [9]). 
The  test  problems  come  from  the  MacMPEC  test  suite  by  Leyffer  ([21]) 
and  the  set  described  in  [5]  by  Ferris,  and  all  were  formulated  in  AMPL. 
All  testing  was  conducted  on  a  Dell  Workstation  running  Red  Hat 
Linux  9  with  5GB  of  main  memory  and  864MHz  clock  speed.  Results  of 
comparative  testing  are  provided  in  the  tables  below.  This  information 
is  intended  to  display  the  relative  robustness  of  the  codes.  Runtimes 
are  not  included  in  the  comparison  because  of  two  reasons:  (1)  Most 
of  the  problems  are  solved  in  under  0.01  CPU  seconds  by  both  solvers, 
and  (2)  the  solvers  were  compiled  using  different  compilers  (LOQO  with 
GNU  compiler  gcc,  and  filter  using  Intel  compilers)  which  can  lead  to 
significant  differences  in  runtimes.  The  first  set  of  tables  pertain  only  to 
loqo’s  performance  on  the  MacMPEC  set,  as  some  of  the  information 
displayed  there  is  not  relevant  for  the  comparison  to  FILTER.  The  next 
set  of  tables  compare  LOQO  and  FILTER  on  the  MacMPEC  set,  and  the 
last  group  of  tables  show  the  results  on  the  Ferris  set  for  both  solvers. 


MacMPEC  testset  LOQO  results 

problem 

vars 

§1213 

time 

optimal  obj.  fn. 

final  relax 

initial  pen 

final  pen 

bardl 

9 

DHI 

13 

0 

1.700000E+01 

1.38E-09 

1.00E+01 

1.00E+01 

bardlm 

M 

3 

wm 

0 

1.700000E+01 

2.26E-09 

1.00E+01 

1.00E+01 

bard2 

H 

KB 

■jUB 

0 

6.598000E+03 

3.46E-11 

1.00E+03 

1.00E+03 

bard2m 

16 

12 

21 

0 

-6.598000E+03 

5.43E-10 

1.00E+03 

1.00E+03 

bard3 

8 

6 

19 

0 

-1.267871E+01 

5.18E-10 

1.00E+01 

.1.00E+01 

bard3m 

wm 

8 

on 

-1.267871E+01 

3.99E-10 

1.00E+01 

1.00E+01 

wm 

40 

29 

.01 

1.016657E+04 

2.51E-13 

1.00E+04 

1.00E+04 

bem-milanc30-s 

FEW 

127 

59.2 

9.443495E-02 

1.30E-11 

bilevl 

EEB 

wm 

wm 

0 

1.117272E-09 

1.19E-13 

1.00E+02 

bilev2 

wm 

mm 

.01 

2.79E-10 

1.00E+03 

1.00E+03 

bilevel  1 

17 

15 

37 

.01 

3.851617E-09 

1.08E-12 

1.00E+02 

1.00E+02 

bilevel2 

25 

21 

22 

.01 

-6.600000E+03 

2.97E-10 

bilevel3 

wm 

13 

Efl 

0 

-1.267871E+01 

1.03E-11 

1.00E+02 

1.00E+02 

bilin 

keb 

EB 

mm 

0 

1.460000E+01 

1.85E-12 

1.00E+02 

1.00E+02 

dempe 

0 

3.125001E+01 

2.93E-11 

1.00E+01 

1.00E+01 

design- cent- 1 

16 

14 

EBI 

1.860647E+00 

1.37E-11 

design-cent-2 

KB 

18 

wm 

3.483816E+00 

1.15E-10 

1.00E+01 

1.00E+01 

design-cent-3 

mm 

EB 

.02 

3.723370E+00 

7.88E-14 

design-cent-4 

31 

28 

26 

.01 

-2.130414E-09 

2.22E-10 

desilva 

9 

6 

wm 

DH 

-1.000000E+00 

2.45E-10 

l.OOE-hOO 

1.00E+00 

dfl 

4 

4 

wm 

3.109960E-09 

1.15E-10 

1.00E+01 

1.00E+01 

ex9.1.1 

19 

17 

m 

OH 

-1.300000E+01 

4.98E-10 

1.00E+02 

1.00E+02 
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problem 


ex9.1.10 


ex9.1.2 


ex9.1.3 


ex9.1.4 


ex9.1.5 


ex9.1.6 


ex9.1.7 


ex9.1.8 


ex9.1.9 


ex9.2.1 


ex9.2.3 


ex9.2.4 


ex9.2.5 


ex9.2.6 


ex9.2.8 


ex9.2.9 


flp2 


flp4-l 


flp4-2 


flp4-3 


np4-4 


gauvin 


gnashlO 


gnashll 


gnash  12 


gnash  13 


gnash  14 


gnashl5 


gnashl6 


gnash  17 


hakonsen 


hs044-i 


incid-setl-16 


incid-setl-32 


incid-setl-8 


incid-setlc-16 


incid-setlc-32 


incid-setlc-8 


incid-set2-32 


incid-set2-8 


incid-set2c-16 


incid-set2c-32 

incid-set2c-8 


jrl 


jr2 


kthl 


kth2 


kth3 


liswet  1-050 


liswet  1-100 


liswet  1-200 


15  12 


18  16 


MacMPEC 


iters  time 


12  0 


0 


37  .01 


15  0 


13  0 


28  .01 


.01 


0 


17  0 


14  0 


42  0 


17  0 


0 


14  0 


24 


301  350 


6  4 


22 


22 


22 


22 


22 

22 


22  20 


22 


14  12 


31  24 


711  716 


711  731 


295 


6.89E-11 


1.25E-09 


1.16E-09 


1.73E-10 
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optimal  obj.  fh.  final  relax 


-3.250000E+00  1.25E-09 


-6.250000E+00  1.32E-09 


-2.920000E+01  1.27E-10 


-3.700000E+01  5.45E-10 


-1.000000E+00  1.08E-10 


5  i^niiiiiiin  oggiil 


-2.600000E+01 


-3.250000E+00 


3.111111E+00 


1.700000E+01 


9.999167E+01 


5.000000E+00 


5.000000E-01 


9.000000E+00 


-1.000000E+00 


1.700000E+01 


1.500000E+00 


2.000000E+00 


4.150241E-09 


5.933353E-09 


7.093403E-10 


1.881357E-09 

2.386228E-08 


4.34E-11 


2.72E-09 


5.30E-11 


6.71E-10 


1.73E-10 


8.35E-11 


3.54E-11 


4.15E-10 


1.12E-11 


1.07E-12 


2.95E-13 
"  4.61E-13 


-2.308232E+02 

-1.299119E+02 


-3.693311E+01 


-7.061783E+00 


-2.414420E-I-02 


-9.074910E+01 


-2.569821E+01 


-6.116708E+00 


2.436681E+01 


1.561777E+01 


4.467425E-09 


3.058662E-07 


7.140892E-09 


3.684200E-09 


9.952331E-06 


1.286690E-09 


3.216853E-03 


1.736491E-03 


EirtirtftwagiKl 


3.599585E-03 


2.435879E-03 

5.471291E-03 


5.000000E-01 


IHH 


1.207329E-08 

2.707954E-08 


5.000000E-01 


1.399429E-02 

1.373395E-02 


1.700859E-02 


9.99E-13 

5.53E-12 


4.58E-13 


9.47E-14 


5_ 

2 


4.51E-11 


1.60E-11 


1.84E-12 


1.06E-11 


2.35E-12 


4.56E-12 


2.71E-11 


1.96E-12 


6.68E-11 


5.65E-12 


1.27E-12 


1.69E-12 


1.15E-12 


4.52E-12 

1.85E-11 


3.57E-09 


5.71E-09 

2.42E-08 


2.94E-09 


1.31E-11 


3.17E-12 


4.99E-13 


final  pen 

CTi!i!5ggini 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+01 

1.00E+01 

1.00E+01 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+03 

jQjQSSiS 

1.00E+02 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+01 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+01 

1.00E+01 

1.00E+01 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+03 

1.00E+03 

1.00E+04 

1.00E+04 

1.00E+01 

1.00E+01 

1.00E+03 

1.00E+03 

IHiMES 

1.00E+03 

1.00E+03 

1.00  E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+01 

1.00E+02 

1.00E+03 

1.00E+03 

1.00E+03 

1.00  E+03 

1.00E+02 

1.00E+02 

1.0GE+03 

1.00E+03 

1.00E+03 

1.00E+02 

1.00E+02 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+02 

1.00E+02 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+03 

1.00E+02 

1.00E+02 

1.00E+00 

1.00E+00 

1.00E+00 

1.00E+01 

1.00E+00 

1.00E+00 

1.00E+00 

1.00E+00 

1.00E+00 

1.00E+01 

1.00E+01 

1.00E+01 

1.00E+02 

1.00E+02 

1.00E+02 

1.00E+02 
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MacMPEC  test  set  LOQO  results 


problem 

vars 

cons 

iters 

time 

optimal  obj.  fh. 

final  relax 

initial  pen 

final  pen 

nashl 

9 

6 

15 

0 

1.014401E-09 

UUaidH 

1.00E+01 

1.00E+01 

outrata31 

10 

8 

13 

0 

3.207700E+00 

1.00E+01 

1.00E4-01 

outrata32 

mm 

8 

13 

0 

3.449404E+00 

4.20E-10 

1.00E+01 

outrata33 

10 

8 

12 

0 

4.604254E+00 

1.03E-09 

1.00E+01 

1.00E+01 

m 

8 

Dm 

0 

6.592684E+00 

7.83E-09 

1.00E+01 

1.00E+01 

19.68 

6.169515E-01 

1.21E-10 

1.00E+01 

1.00E+01 

KiTifcl 

70.95 

6.529792E-01 

1.51E-09 

1.00E+01 

1.00E+01 

pack-comp  1-8 

170 

77 

.99 

ummamu 

1.47E-10 

1.00E+01 

pack-complc-16T 

693 

751 

38 

2.22 

1.02E-10 

1.00E+01 

1.00E+01 

■Mima 

Hi» 

48.74 

6.614431E-01 

2.35E-10 

1.00E+01 

1.00E+01 

177 

.41 

6.000000E-01 

1.75E-10 

1.00E+01 

1.00E+01 

pack-complp-16 

693 

691 

152 

13.19 

6.169500E-01 

2.73E-14 

1.00E+06 

1.00E+06 

pack-complp-32 

2917 

2915 

386 

186.85 

1.59E-14 

1.00E+06 

1.00E+06 

155 

urn 

9.05 

4.01E-16 

1.00E+06 

1.00E+06 

|  pack-comp2-16 

693 

736 

22.17 

7.271355E-01 

6.71E-10 

1.00E+01 

1.00E+01 

2917 

3068 

106 

47.10 

7.826040E-01 

2.86E-10 

1.00E+01 

1.00E+01 

pack- comp  2- 8 

■BiM 

55 

.72 

6.731172E-01 

1.42E-10 

1.00E+01 

1.00E+01 

Hggf 

iggy 

29 

1.65 

7.274676E-01 

1.11E-09 

1.00E+01 

l.OOE+Ol 

|  pack-comp2c-32t 

2917 

78 

30.40 

7.829438E-01 

1.58E-10 

1.00E+01 

H 

EM 

.27 

6.734582E-01 

3.05E-10 

1.00E+01 

l.OOE+Ol 

|  pack-comp2p-16 

693 

BSHM 

■EF:M 

11.46 

7.271354E-01 

2.28E-14 

1.00E+06 

1.00E+06 

2917 

2915 

83.20 

7.826040E-01 

1.48E-15 

1.00E-I-06 

1.00E+06 

pack-comp2p-8 

157 

155 

.86 

6.731150E-01 

1.00E+06 

1.00E+06 

pack-rigl-16 

539 

537 

13.91 

8.260127E-01 

1.00E-09 

1.00E+01  i 

1.00E+01 

pack-rigl-32T 

2331 

2329 

216 

72.57 

8.508944E-01 

3.40E-08 

1.00E+01 

1.00E+01 

pack-rigl-8 

mm 

.41 

7.879318E-01 

1.91E-10 

1.00E+01 

1.00E+01 

pack-riglc-16 

539 

552 

•mm 

8.264985E-01 

1.87E-09 

1.00E+01 

1.00E+01 

2331 

mi 

83 

\mjsm 

8.516407E-01 

1.11E-09 

1.00E+01 

1.00E+01 

pack-riglc-8 

■WiM 

28 

.21 

7.883002E-01 

1.63E-09 

1.00E+01 

1.00E+01 

pack-riglp-16 

mm 

42 

2.16 

8.260127E-01 

9.17E-14 

1.00E+05 

1.00E+05 

2717 

BM.-M 

193.60 

8.508944E-01 

3.86E-08 

1.00E+01 

1.00E+01 

|  pack-riglp-8 

153 

151 

.23 

7.879318E-01 

5.06E-14 

1.00E+04 

1.00E+04 

525 

523 

IL 

1.38E-04 

1.00E+01 

1.00E+06 

E22I 

IL 

-5.95E+05 

1.00E+01 

1.00E+01 

|  pack-rig2-8 

■MiJ 

114 

29 

.24 

5.06E-10 

1.00E+01 

1.00E+01 

|[?E22535iE£SBB 

525 

538 

IL 

2.33E-02 

1.00E+01 

1.00E+01 

2242 

2271 

IL 

1.83E-04 

1.00E+01 

1.00E+01 

pack-rig2c-8 

116 

121 

29 

.22 

7.993058E-01 

3.06E-10 

1.00E+01 

1.00E+01 

pack-rig2p-16 

629 

51 

1.085137E+00 

6.79E-15 

1.00E+05 

1.00E+05 

2621 

253 

83.62 

1.135885E+00 

1.61E-13 

1.00E+05 

1.00E+05 

pack-rig2p-8 

Esm 

147 

EH 

.24 

7.804043E-01 

1.11E-13 

1.00E+04 

1.00E+04 

pack-rig3-16 

526 

■HIM 

10.39 

8.004305E-01 

5.03E-10 

1.00E+01 

1.00E+01 

pack-rig3-32 

E 

jtwtii 

|g| 

173.37 

8.864084E-01 

2.18E-09 

1.00E+01 

l.OOE+Ol 

pack-rig3-8 

116 

114 

eh 

.23 

7.352021E-01 

3.13E-11 

1.00E+01 

1.00E+01 

pack-rig3c-16 

528 

541 

1  312 

18.66 

8.186004E-01 

2.53E-10 

1.00E4-01 

1.00E+01 
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MacMPEC  testset  LOQO  results  | 

problem 

cons 

iters 

time 

optimal  obj.  fn. 

final  relax 

initial  pen 

final  pen 

pack-rig3c-32T 

2248 

mm 

26.57 

9.089097E-01 

1.00E-09 

l.OOE-fOl 

1.00E+01 

pack-rig3c-8 

116 

MU 

EH 

7.534726E-01 

1.39E-10 

l.OOE-f  01 

l.OOE-fOl 

portfl-i-1 

100 

mm 

30 

EH 

3.16E-10 

l.OOE-fOl 

l.OOE-f  01 

Eul 

lE^H 

i» 

3.78E-11 

1.00E+01 

1.00E+01 

■hum 

mm 

'mm 

.04 

6.279030E-06 

1.35E-10 

l.OOE-f  01 

1.00E+01 

portfl-i-4 

m 

37 

\  28 

.04 

2.186321E-06 

1.26E-10 

l.OOE-f  01 

l.OOE-fOl 

portfl-i-6 

■IiM 

llwi 

IEOH 

.03 

2.364383E-06 

5.01E-11 

1.00E+01 

l.OOE-fOl 

qpecl 

EM 

KB 

mm 

0 

8.000000E+01 

1.36E-09 

l.OOE-f  02 

1.00E+02 

qpec- 100-1 

W'fsM 

EM 

EM 

mm 

9.900279E-02 

1.52E-10 

1.00E+02 

1.00E+02 

mm 

-6.44521 1E+00 

1.16E-10 

1.00E+02 

1.00E-f02 

qpec- 100-3 

211 

26 

-5.481671E+00 

1.26E-10 

1.00E+02 

l.OOE-f  02 

qpec- 100-4 

mwm 

»« 

WiEM 

-4.0585 18E+00 

3.62E-11 

l.OOE-f  02 

l.OOE-f  02 

qpec2 

Em 

40 

KE9 

.04 

4.499392E+01 

2.67E-08 

1.00E+02 

1.00E-H05 

qpec-200-1 

EH 

404 

74 

14.19 

6.13E-12 

1.00E+02 

1.00E+03 

qpec-200-2 

19 

3.88 

-2.403634E+01 

2.79E-10 

1.00E+02 

l.OOE-f  02 

421 

408 

56 

10.28 

-1.953406E+00 

6.69E-12 

1.00E+02 

l.OOE-f  03 

|  qpec- 200-4 

441 

wMm 

35 

-6.230523E+00 

1.00E+02 

1.00E+02 

4 

2 

84 

2.50E-07 

1.00E+00 

l.OOE-f  03 

IL 

1.57E+04 

l.OOE-f  00 

l.OOE-f  00 

■HIM 

ES3 1 

■E1:M 

12.19 

-5.174566E+02 

2.63E-07 

1.00E+03 

1.00E+03 

scholtesl 

5 

2 

mm 

.01 

2.000000E+00 

1.01E-09 

l.OOE-fOl 

l.OOE-fOl 

scholtes2 

5 

2 

0 

1.500000E+01 

1.69E-09 

l.OOE-fOl 

l.OOE-f  01 

scholtes3 

4 

2 

33 

0 

5.000000E-01 

1.84E-09 

l.OOE-f  00 

l.OOE-fOl 

scholtes4*r 

5 

4 

96 

.01 

l.OOE-f  00 

l.OOE-f  03 

scholtes5 

6 

4 

10 

.01 

1.000000E+00 

5.05E-09 

l.OOE-f  00 

l.OOE-f  00 

siouxfls 

4152 

4124 

72 

85.62 

2.080014E+02 

6.92E-14 

!  l.OOE-f  04 

l.OOE-f  04 

siouxflsl** 

4152 

4124 

IL 

-1.75E-03 

l.OOE-f  04 

l.OOE-f  04 

sll 

8 

25 

0 

2.08E-09 

l.OOE-f  00 

l.OOE-f  00 

stackelbergl 

5 

3 

0 

-3.266667E-03 

5.87E-10 

1.00E+02 

1.00E+02 

119 

Ti  !<■ 

114 

.33 

1.102311E+02 

2.71E-14 

1.00E+03 

1.00E+03 

278 

250 

55 

1.85 672 1E+02 

4.09E-14 

l.OOE-f  03 

l.OOE-f  03 

EBEBBSiBI 

258 

204 

IL 

2.23E-07 

1.00E+03 

l.OOE-f  03 

water-net1 1 

81 

64 

IL 

4.14E-01 

l.OOE-f  02 

l.OOE-f  02 

-fmeans  the  starting  slack  values  in  LOQO  had  to  be  set  to  .1,  except  for  bar-truss-3, 
for  which  they  were  set  to  100 

-*  means  the  problem  was  listed  as  infeasible  in  the  original  test  set 

-  **  means  the  problem  is  rank-deficient,  a  common  source  of  difficulty  for  interior-point 
methods 

-  **  :  the  initial  estimate  for  the  penalty  parameter  was  l.OOE-f  05,  for  which  LOQO  was 
unable  to  obtain  a  solution;  the  solution  was  obtained  by  manually  setting  the  penalty  at 
10  and  setting  starting  slacks  to  .1 

-  as  described  before,  this  problem  is  unbounded  for  penalty  <  2;  LOQO  solves  it 
easily  if  the  penalty  is  set  to  be  greater  than  2 

means  LOQO  had  trouble  obtaining  accuracy  on  the  problem  as  the  solution  it  was 
trying  to  find  is  not  a  KKT  point 
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MacMPEC:  LOQO  vs.  FILTER 

LOQO  j 

FILTER  | 

problem 

iters  1 

optimal  obj.  fn. 

iters 

optimal  obj.  fn. 

bardl 

EBB 

1.700000E+Oi 

3 

1.700000E+01 

bardlm 

EBB 

1.700000E+01 

3 

1.700000E+01 

bard2 

EMI 

6.598000E+03 

1 

6.598000E+03 

bard2m 

EM 

EBfll 

bard3 

EDB 

EBU 

EKBial 

bard3m 

EM 

il 

-1.267871E+01  | 

bar-truss-3T 

29 

1.016657E+04  | 

EM 

bem-milanc30-s 

127 

IsElklkMafiiltHI 

1.298310E-01  | 

bilev  1 

EM\ 

BSM 

bilev2 

22 

U 

-6.600000E+03  | 

bilevell 

3.851617E-09  | 

EM 

KTiTiTiTiTiTiTwaiiMI 

bilevel2 

7 

-6.600000E+03 

bilevel3 

EM 

7 

-1.267871E+01 

bilin 

EM 

1.460000E+01 

n 

5.466670E+00 

dempe 

k m 

3.125001E+01 

1 

3.125000E+01 

design-cent-1 

4 

1.860650E+00 

design-cent-2 

EM 

3.483816 

INF 

design-cent-3 

24 

3.723370E+00 

rho  <  eps 

design-cent-4 

26  J 

4 

3.079200E+00 

desilva 

EM 

-1.000000E+00 

2 

-1.000000E+00 

dfl 

EM 

2 

O.OOOOOOE+OO 

ex9.1.1 

KEH 

-1.300000E+01 

1 

ex9.1.10 

12 

-3.250000E+00 

1 

-3.250000E+00 

ex9.1.2 

13 

-6.250000E+00 

3 

-6.250000E+00 

ex9.1.3 

37 

3 

-2.920000E+00 

ex9.1.4 

EBB 

2 

-3.700000E+01 

ex9.1.5 

EM 

-1.000000E+00 

3 

-1.000000E+00 

ex9.1.6 

28 

EM 

-2.100000E+01 

ex9.1.7 

29 

|  -2.600000E+01 

Li _ 

IKuK'i'rrrtiaEstiH 

ex9.1.8 

12 

m 

|  -3.250000E+01  | 

ex9.1.9 

17 

|  3.111111E+00 

am 

ex9.2.1 

\EM 

IQ 

|  1.700000E+01  | 

ex9.2.2*f 

|  42 

9.999167E+01 

22 

ex9.2.3 

ie m 

5.000000E+00 

1 

-5.500000E+01 

ex9.2.4 

Hi 

5.000000E-01 

OB 

5.000000E-01 

ex9.2.5 

IEEB 

EM 

ex9.2.6 

23 

mmmsm 

EM 

-1.000000E+00 

ex9.2.7 

14 

|  1.700000E+01 

3 

1.700000E+01 

ex9.2.8 

3 

IEE3M3EEB 

3 

1.500000E+00 

ex9.2.9 

13 

|  2.000000E+00 

IEM 

2.000000E+00 

flp2 

IEEB 

3 

flp4-l 

|  37 

5.933353E-09 

3 

2.77635E-30 

flp4-2 

IEEB 

7.093403E-10 

3 

O.OOOOOOE+OO 

flp4-3 

IEB3 

1.881357E-09 

1.191820E-30 

flp4-4 

|  65 

in 

5.896360E-31 

IPMs,  Penalty  Methods,  Equilibrium  Problems 
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|  MacMPEC:  loqo  vs.  filter  | 

\  LOQO 

FILTER 

problem 

optimal  obj.  fn. 

iters 

optimal  obj.  fh. 

gauvin 

2.000000E+01 

3 

2.000000E+01 

eh 

-2.308232E+02 

8 

-2.308230E+02 

Em 

-1.299119E+02 

8 

-1.299120E+02 

gnash  12 

ESH 

-3.69331 1E+01 

9 

-3.693310E+01 

gnash  13 

e m 

-7.061783E+00 

13 

-7.061780E+00 

gnash  14 

E9H 

-1.790463E-01 

IEH 

-1.790460E-01 

gnash  15 

EBI 

-3.546991E+02 

^8 

-3.546990E+02 

gnash  16 

-2.414420E+02 

EH 

24 

-9.074910E+01 

EH 

-9.074910E+01 

rziMmmm 

21 

-2.569821E+01 

15 

-2.569820E+01 

21 

— 

10 

hakonsen 

28 

2.436681E+01 

10 

2.436680E+01 

hs044-i 

mm 

1.561777E+01 

6 

1.561780E+01 

incid-setl-16 

4.467425E-09 

33 

1.740230E-01 

incid-setl-32 

353 

3.058662E-07 

IL 

incid-setl-8 

7.140892E-09 

34 

2.320310E-01 

iBMM 

3.684200E-09 

34 

m\i\  1  III 

incid-setlc-32 

243 

9.952331E-06 

IEH 

incid-setlc-8 

26 

1.286690E-09 

2.320310E-01  | 

incid-set2-16 

63 

3.216853E-03 

incid-set2-32 

1.736491E-03 

1.750980E-03 

incid-set2-8 

32 

4.517882E-03 

48 

4.517880E-03 

incid-set2c-16 

54 

3.599585E-03 

37 

3.678050E-03 

incid-set2c-32 

2.435879E-03 

41 

2.451 130E-03 

incid-set2c-8 

'whm 

5.471291E-03 

24 

jrl 

mm 

5.000000E-01 

1 

5.000000E-01 

jr2 

35 

5.000000E-01 

7 

5.000000E-01 

EH 

1.207329E-08 

1 

O.OOOOOOE+OO 

|g| 

2.707954E-08 

tm 

O.OOOOOOE+OO 

kth3 

5.000000E-01 

4 

liswet  1-050 

20 

1.399429E-02 

1 

liswet  1-100 

22 

1.373395E-02 

1 

liswet  1-200 

26 

1.700859E-02 

1 

1.700860E-02 

nashl 

15 

1.014401E-09 

5 

0.000000E+00 

outrata31 

13 

3.207700E+00 

8 

3.207700E+00 

outrata32 

13 

3.449404E+00 

8 

3.449400E+00 

outrata33 

12 

4.604254E+00 

7 

mmmm\ 

outrata34 

11 

6.592684E+00 

6 

6.592680E+00 

pack-comp  1-1 6 

187 

6.169515E-01 

EH 

6.169510E-01 

pack-compl-32T 

21 

6.529790E-01 

6.000000E-01 

8 

6.000000E-01 

|  pack-complc-16T 

38 

6.230428E-01 

5 

6.230430E-01 

108 

6.614431E-01 

13 

6.614430E-01 

pack-complc-8 

35 

6.000000E-01 

8 

6.000000E-01 

pack-comp  lp- 16 

152 

6.169500E-01 

EH 

6.169510E-01 

6.529793E-01 

EH 

6.529770E-01 

5.999999E-01 

53 

6.000000E-01 

pack-comp2-16 

■fckM 

7.271355E-01 

43 

7.271350E-01 

106 

7.826040E-01 

E— 1 

pack-comp2-8 

55 

6.731172E-01 

8 

6.731170E-01 

29 

7.274676E-01 

15 

7.274680E-01 

78 

7.829438E-01 

7 

7.829440E-01 
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MacMPEC:  LOQO  vs.  filter  j 

LOQO 

FILTER  j 

problem 

optimal  obj.  fn. 

iters 

EH 

6 

128 

32 

7.271350E-01 

pack-comp2p-32T 

ee 

7.826040E-01 

47 

7.826040E-01 

pack-comp2p-8 

6.731150E-01 

6.731170E-01 

pack-rigl-16 

ebb 

8.260127E-01 

64 

8.260130E-01 

216 

8.508944E-01 

8.508950E-01 

pack-rigl-8 

EM 

7.879318E-01 

7 

7.879320E-01 

pack-riglc-16 

■liW 

8.264985E-01 

KH 

8.264980E-01 

83 

8.516407E-01 

KH 

8.516410E-01 

pack-riglc-8 

EH 

7.883002E-01 

6 

7.883000E-01 

pack-riglp-16 

eh 

8.260127E-01 

28 

2.644980E+02 

435 

8.508944E-01 

125 

2.242060E+03 

|  pack-riglp-8 

28 

7.879318E-01 

14 

3.594420E+01 

IL 

INF 

IL 

INF 

|  pack-rig2-8 

29 

7.804043E-01 

IL 

6 

1.000000E+00 

IL 

11 

1.000000E+00 

pack-rig2c-8 

EH 

7.993058E-01 

6 

7.993060E-01 

pack-rig2p-16 

EM 

1.085137E+00 

6.259330E+02 

pack-rig2p-32t 

253 

20 

8.717450E+02 

pack-rig2p-8 

7.804043E-01 

20 

4.667880E+01 

pack-rig3-16 

8.004305E-01 

32 

8.004310E-01 

pack-rig3-32 

time  limit 

pack-rig3-8 

7.352021E-01 

KH 

7.352020E-01 

pack-rig3c-16 

EH 

8.186000E-01 

pack-rig3c-32* 

99 

9.089097E-01 

14 

9.089100E-01 

EE 

7.534726E-01 

7 

7.534730E-01 

portfl-i-1 

30 

1.503723E-05 

5 

1.502420E-05 

portfl-i-2 

29 

4 

1.457280E-05 

portfl-i-3 

iH 

4 

6.26499E-06 

portfl-i-4 

Efl 

2.186321E-06 

4 

2.177340E-06 

portfl-i-6 

ee 

2.364383E-06 

4 

qpecl 

KH 

8.000000E+01 

3 

8.000000E+01 

qpec-100-1 

ee 

7 

9.900280E-02 

qpec- 100-2 

KH 

7 

eh— 

qpec- 100-3 

Efl 

6 

-5.482870E+00 

qpec- 100-4 

ee 

ESUdMkiaSlM 

-3.672010E+00 

qpec2 

108 

2 

4.500000E+01 

qpec- 200-1 

74 

KH 

-1.934830E+00 

qpec- 200- 2 

KH 

-2.403634EH-01 

KH 

-2.407740E+01 

qpec-200-3 

ee 

-1.953406E+00 

KH 

-1.953410E+00 

qpec- 200-4 

35 

5 

ralphl*T 

84 

-2.500110E-04 

27 

HMWH 

mmmmm 

IL 

11 

-6.256730E-07 

ralphmod 

148 

7 

-5.025770E+02 

scholtesl 

10 

2.000000E+00 

4 

2.000000E+00 

scholtes2 

11 

1.500000E+01 

1.500000E+01 

scholtes3 

ee 

4 

5.000000E-01 

bsshi 

96 

26 

-2.541610E-07 

scholtesS 

KH 

1.000000E+00 

1 

1.000000E+00 

siouxfls 

Efl 

2.080014E+02 

10 

2.082580E+02 

siouxflsl** 

IL 

inf  QP 

sll 

EE 

1.000093E-04 

1 

1.000000E-04 

stackelbergl 

KH 

-3.266667E-03 

4 
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MacMPEC:  LOQO  VS.  FILTER  1 

LOQO 

FILTER 

problem 

iters 

optimal  obj.  fn. 

iters 

optimal  obj.  fh. 

tap-09 

DEI 

iTiWtlllaSliV 

21 

1.091310E+02 

28 

1.876320E+02 

I25EH3HI 

IL 

3.475600E+03 

IL 

time  limit 

38 
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In  addition  to  the  problem  documented  in  Section  5,  the  penalty 
method  was  able  to  solve  some  problems  which  a  standard  implemen¬ 
tation  of  LOQO  was  unable  to  solve.  These  problems  were  ex9.2.2 , 
pack-rig2-16,  pack-rig2p-32 ,  qpec2,  ralphl,  scholtes4,  tap-09,  and  tap- 
15. 

B.l.  Ferris  Problems  (pure  complementarity  problems). 


problem 

loqo  result 

comment 

filter  result 

cammcp 

fail 

non-differentiable  (see  below) 

fail 

duopoly 

solved 

solved 

ehl_kost 

solved 

when  initial  slacks  set  to  .01 

solved 

electric 

error 

error  given  by  AMPL  during  presolve 

AMPL  error 

forcedsa 

solved 

fail 

games 

solved 

fail 

lincont 

solved 

when  all  variables  set  to  1  (see  below) 

solved 

IBM 

fail 

non-differentiable  (see  below) 

fail 

shubik 

solved 

solved 

simple-ex 

solved 

when  variables  x\  —  x\  initialized  at  1  (see  below) 

solved 

spillmcp 

solved 

solved 

1  •  1 

C UjfllUULsjJ)  jjyuvtlj±  KJV  .  O-  ^  - - - - 

are  not  differentiable  at  a  point,  a  known  sources  of  trouble  for 
interior-point  methdods.  Hence,  LOQO’s  failure  on  these  two 
problems  does  not  seem  to  be  due  to  their  being  equilibrium 
problems  as  such. 

ehLkost,  simple-ex:  If  nonnegative  variables  are  initialized  at  0 
or  not  initialized  at  all  (in  which  case  LOQO  automatically  sets 
them  to  0),  an  interior-point  method  can  have  trouble.  It  is 
therefore  understandable  that  loqo’s  success  on  these  problems 
depends  on  the  variables  in  question  being  initialized  away  from 
0. 
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